Autonomic Runtime Manager for Large Scale Adaptive Distributed Applications

Large-scale distributed applications are highly adaptive and heterogeneous in terms of their computational requirements. The computational complexity associated with each computational region or domain varies continuously and dramatically both in space and time throughout the whole life cycle of the application execution. Consequently, static scheduling techniques are inefficient to optimize the execution of these applications at runtime. In this paper, we present an Autonomic Runtime Manager (ARM) that uses the application spatial and temporal characteristics as the main criteria to selfoptimize the execution of distributed applications at runtime. The wildfire spread simulation is used as a running example to demonstrate the ARM effectiveness to control and manage the application’s execution. The behavior of the wildfire simulation depends on many complex factors that contribute to the adaptive and heterogeneous behaviors such as fuel characteristics and configurations, chemical reactions, balances between different modes of heat transfer, topography, and fire/atmosphere interactions. Consequently, the application execution cannot be predicted a priori and that makes static parallel or distributed algorithms very inefficient. The ARM is implemented using two modules: 1) Online Monitoring and Analysis Module, and 2) Autonomic Planning and Scheduling Module. The online monitoring and analysis module interfaces with different kinds of application and system sensors that collect information to accurately determine the current state of the fire simulation in terms of the number and locations of burning and unburned cells as well as the states of the resources, and decides whether the autonomic planning and scheduling module should be invoked. The autonomic planning and scheduling module uses the resource capability models as well as the current state of the computations to repartition the whole computational workload into available processors. Our experimental results show that by using ARM the performance of the wildfire simulation has been improved by 45% when compared with a static partitioning algorithm. We also evaluate the performance of ARM using two partitioning strategies. One approach is to partition the wildfire simulation domain into Natural Regions (NR), where each region has the same temporal and spatial characteristics (e.g., burned (NR1), burning (NR2), and unburned regions (NR3)), and schedule each region into available processors. The second approach is to view the wildfire domain as a graph and use a graph partitioning tool (e.g., ParMetis tool) to partition the graph into different domains.